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ABSTRACT 

The  use  of  branch-and-bound  methods  to  reduce  the  storage  require- 
ments in  discrete  dynamic  programs  is  discussed.   Relaxations  and  fathoming  criteria 
are  used  to  eliminate  states  whose  corresponding  subpolicies  could  not  lead 
to  optimal  policies.   The  resulting  combined  dynamic  programming /branch- and  - 
bound  algorithms  are  applied  to  several  typical  discrete  decision  problems. 
Including  the  traveling  -salesman   problem  and  the  nonlinear  knapsack  problem  • 
Computational  experience,  demonstrating  the  dramatic  savings  in  both  computer 
storage  and  computational  requirements  which  were  effected  utilizing  this 
combined  approach,  is  also  reported. 


1.   Introduction 

Consider  the  following  functional  equations  of  a  dynamic  pro- 
gramming algorithm: 

f(y)  =  min  ih(f(y'),  y' .  d)|  T(y',d)  =  y]  ,     y  t(ii  -  y^)      (1) 
with  the  boundary  condition 

f(yo)  =  ?o  (2) 

in  which,  il   is  the  finite  nonempty  state  space;  y.  €  Q  is  the  initial  state; 
d  €  D  is  a  decision,  where  D  is  the  finite  nonempty  set  of  decisions; 
T:  Q  X  D  -•  Q  is  the  transition  mapping,  where  T(y  ,d)  is  the  state  that  is 
reached  if  decision  d  is  applied  at  state  y  ;h:IRxnxD-']Ris  the  (monotone) 
cost  function,  where  h(5,y  ,d)  is  the  cost  of  reaching  state  T(y  ,d)  €  Q  by 
an  initial  sequence  of  decisions  that  reaches  state  y  6  Q  at  cost  §  €  ]R  and 
is  then  extended  by  decision  d;  §  €  IR  represents  the  initial  cost  incurred 
in  the  initial  state  y_  t  Q;  and  In  (2)  we  make  the  convenient,  but  unnecessary, 
assumption  that  return  to  the  initial  state  is  not  possible.   The  functional  equa- 
tions (1)  and  (2)  are  simply  mathematical  transliterations  of  Bellman's  principle 
of  optimality  [l]- 

The  recursive  solution  of  these  functional  equations  to  determine 
the  cost  of  an  optimal  solution  and  the  subsequent  (policy)  reconstruction  pro- 
cess to  determine  the  set  of  optimal  sequences  of  decisions  is  straight- 
forward.  However,  the  evaluation  of  f(y)  by  (1)  necessitates  access  to 
f(y')  in  high-speed  (magnetic  core,  thin  film)  computer  storage  for  all 
states  [y' I  T(y',d)  =  y}  and  the  policy  reconstruction  process  necessitates 
access  to  the  decisions  at  state  y'  which  result  in  f(y)  for  all  states 
in  the  state  space  Q  in  low-speed  (tape, disk)  computer  storage.   It  is  common 
knowledge  that  in  real  problems  excessive  high-speed  storage  requirements 


can  present  a  serious  implementation  problem,  so  much  so  that  many  problems 
cannot  be  solved  even  on  the  largest  present  day  computers. 

This  paper  presents  a  general  approach  to  reducing  both  the  high- 
speed and  the  low-speed  computer  storage  requirements  of  dynamic  programming 
algorithms.   In  order  to  accomplish  this  we  invoke  some  of  the  elementary, 
but  powerful ,  techniques  of  branch-and-bound.   In  particular  we  demonstrate 
that  in  the  evaluation  of  f{y)  the  costly  procedure  of  storing  f(y  )  in  high- 
speed memory  for  all  states  [y  |T(y  ,d)  =  y]  may  not  always  be  necessary. 
Ihat  is,  if  it  can  be  demonstrated  that  there  does  not  exist  a  sequence  of 
decisions  which  when  applied  to  state  y  will  lead  to  an  optimal  solution, 
then  it  is  not  necessary  to  store  f(y  )  in  high-speed  memory  (nor  is  it 
necessary  to  store  the  corresponding  decision    which  led  to  state  y   in  low-speed 
memory).  Such  states  y'  can  be  identified  by  the  use  of  relaxations  and  fathoming 
criteria  which  are  commonly  employed  in  branch-and-bound  and  other  enumera- 
tive  algorithms  --  see  [11,12,21,23,34]  for  example.   Our  approach  has  been 
inspired  by  the  ingenious  bounding  schemes  employed  within  dynamic  programming 
algorithms  in  [29]  and  [35],  by  observations  made  by  the  authors  in  [26],  and 
by  the  computational  success  of  the  resulting  hybrid  algorithm  [22]  --  see  also  [7,28] 

The  outline  of  the  paper  is  as  follows.   Preliminaries  and  definitions 
are  presented  in  S  2.   The  use  of  fathoming  criteria  and  relaxations  within 
dynamic  programming  algorithms  is  then  developed  in  a  general  setting  in  S  3.   The 
versatility  of  the  results  is  manifested  via  application  to  a  number  of  classes 
of   probleiife  in  S  4.   A  numerical  example  of  the  classical  traveling-salesman 
problem  is  presented  and  solved  using  the  results  of  §  3  and  computational 
experience  is  presented  which  demonstrates  the  dramatic  savings  in  both  com- 
puter storage  requirements  and  computation  time  which  were  effected  by  applying 
the  results  of  §  3  to  nonlinear  knapsack  problems.   The  paper  concludes  with 
a  discussion  of  other  possible  applications  and  extensions  S  5. 
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2.   Preliminaries  and  Definitions 


Let  &  =    (D,S,rt)  denote  a  discrete  optimization  problem,  where  D 
is  Che  set  of  all  decisions,  S  c  A  is  the  set  of  feasible  policies,  where  A  is  the 
set  of  all  policies  (strings  or  sequences  of  decisions)  which  can  be  formed  by  concate 
nating  decisions  in  D  and  rr:  S  —IR  is  the  cost  function.  If  %,     6 IR  is  the  cost  of 
an  initial  sequence  of  decisions  6   €  S  and  5~  ^  JR  is  the  cost  of  a  subsequent 
sequence  of  decisions  6.  6  S  then  we  will  find  it  convenient  to  denote  the 
cost  of  S  6   (not  necessarily  in  S)  by  1^(5  ,§„)  =  tt(6  6  ).   (Alternatively, 

we  could  employ  a   composition  operator  "o"  and  write  n(6  5  )  =  tt  (6  ) 
°  TT  (6  )  =  §  °  §«  =  n(§  ,§  )  --  We  shall  interpret  our  results  in  terms  of 
such  returns  in  §  4.1).   Let  tt*  be  the  value  of  an  optimal  solution  to  &, 
i.e.,  tt*  =  min  [rr(6)|6  €  s}  and  let  S*  c  S  be  the  set  of  optimal  solutions  to  ^, 
i.e.  ,  S*  =  [6l6  €  S  A  tt(6)  =  n*}  . 


Assume  that  the  discrete  optimization  problem  ff   is  represented 
[17-19]  by  the  finite  dynamic  program  3   (alternatively,  a  terminating  or 
finite  sequential  decision  process  [6,19]).  ^   is  specified  by  the  triple(  ^^  ,h,S  ) 
where  a  =    (n,D,yQ,T,Qp)  is  a  finite  automaton  [4,30].   Recall  that  Q  is  the 
finite  non-empty  set  of  states,  D  is  the  finite  non-empty  set  of  decisions, 
y^  €  Q  is  the  initial  state,  T:  Q  x  D  -•  Q  is  the  transition  mapping, 
h:]RxnxD-*IRis  the  cost  function  and  §^  6  B.  is  the  cost  incurred  in  the 
initial  state  y^.  Qp  c  Q  is  the  set  of  final  states. 

A  is  the  set  of  all  subpolicies  and  policies  obtained  by  concatenating 
decisions  in  D,   Let  6  C  A  denote  a  subpolicy  or  policy  and  let  e  denote 
the  null  decision,  i.e.   (Vg  6  A),  6e=  &t    =  b  •      The  transition  mapping  T  can 
be  extended  to  Q  x  A  —  D  inductively  by 

(Vy'  €  n),  T(y',e)  =  y',  and 
(Vy'  €  n)(y6  €  A)(Vd  €  D)  ,  T(y',6d)  =  T(T(y' ,6 )  ,d) . 


Slmilarily,  the  cost  function  h  can  be  extended  tolRxQxA-*^ 
inductively  by 

(VE  e  ]R)(Vy'  €  n),  h(5,y',e)  -  F,  and 
dl   €  ]R)(Vy'  €  n)(V6  €  A)  (Vd  €  D)  ,  h(|,y',5d)  =  h(h(^,y' ,§  )  ,T(y ',5  )  ,d) . 
For  y'  €  n  let  A(y  )  denote  the  set  of  feasible  subpolicies  (or  policies  if 
y  €  r2p)which  when  applied  to  y^  result  in  state  y,  i.e.,  A(y')  =  [6  JTCy  ,6  )  =  v') 
let        A*(y  )  C  A(y')  denote  the  set  of  optimal  subpolicies  (or  policies,  if 
y'€  Qp)  for  state  y'  i.e.,  A*(y')  =  [slfCy')  =  hC^^^y^.e)},  and  let  ACOp)  = 
U   /^  „  A(y  )   be  the  set  of  feasible  policies  for  3.      Finally,  we  define  a 
completion  set  of  feasible  subpolicies  for  each  state  y'  6  0  as  x(y')    = 
[6|T(y',6)  f  Qp].   Notice  that  Xiy^)    =  ACQp)  =  the  set  of  strings  accepted 
by  the  automaton  (^. 

The  optimal  value  for  _B   is  f^""  =  min  [f(y)|y  €  Q„]  and  the  set  of 
optimal  policies  for  ^  is  A*  =  [sjf"'^  =  h(^Q'yrv.6 )}  •    A  sufficient  condition 
on  h  to  insure  that  the  dynamic  programming  algorithm  (1)  and  (2)  finds  the 
set  of  optimal  policies  for  the  finite  dynamic  program  3   is  that  h  is  monotone 
[6,19,23,27]  in  the  sense  that 

(ia^,l^)   €  E  xB)(V6  e  A)(Vy'  €  n),  ^^  ^  ^2  ^  ^(l^,y',6)^   h(52'y''6)- 
Finally,  since  j9  is  a  representation  of  ff   we  have  f*  =  n''''  and  a^^  =  ^*' 


3.   Fathoming  Criteria  and  Relaxations 

Let  li   be  an  upper  bound  on  the  value  tt*  of  any  optimal  solution  to  the 
discrete  optimization  problem  ^.   Then,  since  the  finite  dynamic  program  J^ 
is  a  representation  of  P,    it  follows  that  %{   s  f^''.   For  each  state  y'  €  Q 
define  a  (lower  bound)  mapping  i:   Q  -*TR   with  the  property  that 

~(f(y'),Jl(y'))  <  h(f(y'),  y',6)        V^^xCy')    (3) 

Our  main  result  is  stated  in  the  form  of  the  following  theorem. 

Theorem  1.   For  any  y'  €  Q  and  all  6  '5  6  ACClp)  such  that  6'€  A*(y ')  and 
6  €  x(y ') ,  we  have 

n(f(y'),A(y'))  >  r<  ^  6'6  «  S^'  =  A*  (A) 

Proof.   (3)  A  (4)  ^  h(f(y'),  y'.  5)  >  f'  V5  6  xiv) -    D 

Any  state  y'  f  CI  which  satisfies  the  Theorem  is  said  to  be  fathomed 
[12],   If  y'  €  Q  is  fathomed  then  f(y')  does  not  have  to  be  placed  in  high- 
speed memory  and  the  corresponding  last  decision  in  5  '  does  not  have  to  be 
placed  in  low-speed  memory  since  no  feasible  completion  of  6'  can  be  an 
optimal  policy.   That  is,  in  the  evaluation  of  f(y)  with  the  functional 
equations , information  on  previously  fathomed  states  is  irrelevant.   Notice 
also  that  no  feasible  completion  of  any  subpolicy  6 '6  "  €  A  (y"),  where 

T(y',6")  =  y''   i   n„,  can  be  in  A*.   Therefore,  as  a  consequence  of  Theorem 

r 

1,  we  have  the  following  additional  fathoming  criterion: 

Corollary  1.   Let  y'  6  n,  6  '  €  A'ty')  and  5  €  x(y')    satisfy  (4).   Then  all 
successive  states   y"  €  Q  such  that  A(y")  =  [S's'M  T(y',6")  =  y"}  can 
also  be  fathomed  . 
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An  obvious  upper  bound  %{   on  the  cost  n*  of  an  optimal  solution  to 
the  original  optimization  problem^  is  the  cost,  tt(6),  of  any  feasible  solution 
6  6  S  to  P.   The  lower  bounds  i(y  )  can  be  obtained  by  solving  a  relaxed  [12] 
version  of  the  residual  problem  whose  initial  state  is  y' .      If  the  cost 
functions  h  and  rr  are  nonnegative  then  an  immediate  lower  bound  for  any  state 
y  €  n  is  0  in  the  additive  case  and  1  (if  all  coats  are  >    1)  in  the  multiplicative 
case  corresponding  to  a  total  relaxation.  However,  much  better  lower  bounds  can  usually 
be  obtained  by  judicious  choice  of  the  relaxation  as  will  be  demonstrated  in 
the  following  section. 

Notice  that  our  approach  includes  conventional  dynamic  programming 
as  a  special  case  in  which  ^  is  some  sufficiently  large  real  number  and  a 
total  relaxation  is  employed. 
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4.   Examples 

In  this  section  the  results  of  §3are  explicated  and  illustrated 
vls-a-vis  a  number  of  examples.   The  first  example  of  §  4.1  includes  the  second  two 
examples:   the  traveling-salesman  problem  of  §  4.2  and  the  nonlinear  knap- 
sack problem  of  S  4.3,  as  special  cases. 
4.1   Example  1:   Separable  Cost  Functions  [6,23,27]. 

Before  addressing  specific  examples,  we  shall  interpret  the  results  of 
S  3  for  a  general  class  of  problems  which  can  be  approached  with  dynamic 
programming.   Consider  any  discrete  optimization  problem  Q   whose  cost 
function  n       belongs  to  the  class  of  monotone  cost  functions  H  = 
{ttJ  (V6,,62  €  S),  tt(6,62^  = '^;i(6  •,  )on„  (62)!  where  'b  "  represents  a  composition 
operator  [6,23,27].   Specifically,  let  '^  "  denote  any  associative,  commutative, 
isotonic  binary  operator;  common  examples  of  which  are  "+" ,  addition,  "x " , 
multiplication,  and  the  infix  operators  [2]  'V",  disjunction,  and  'X", 
conjunction.   For  any  tt  €  11  and  y  €  Q  we  can  write  (3)  as 

f(y'>i(y')  ^  h(f(y'),  y',6)      Vs  €  ^(y') 
Further,  for  any  n  ^  11  we  can  write  h  as 

h(f(y'),  y',  6)  =  f(y')  o  i(y',6) 

where  I(y',6)  denotes  the  incremental  cost  of  applying  policy  6  at  state  y'. 

Hence,  for  rr  €  11  and  any  y'  €  Q,  (3)  reduces  to 

A(y')  ^  I(y',6)         V6  €  x(y') 
and  Theorem  1  can  be  stated  as  follows  : 

Proposition  1.   I_f  rr  €  n  then  for  any  y'   ^  0.   and  all  6  '6  €  A(Qp)  such 
that  6 '  €  A*(y')  and  ft  €  xiy')   we  have 

f(y')  ol(y')  >  "U  ^  6'6  ^  S*  =  A*. 


-8- 


Wheie  wo  assume  that  f{y  ),  £(y  )  and  !(   are  nonnegative  in  the 
mull  Ipl  icativc  case  ("o"  =  "x"). 

The  remaining  examples  of  this  section  have  cost  functions  belonging 
to  the  additive  subclass  11  Cll,  where  n   =  {tt|(V6  ,6„  €  S),  tt(6  6  )  = 

For  rr  6  H  we  can  write  the  functional  equation  (1)  as 

f(y)  "   min  [l(y',d)  +  f(y')l  T(y',d)  =  y]     y  €(n  -  y^ ) . 

4.2  Example  2:  The  Traveling-Salesman  Problem  [3] 

Let  ^^      m   (v,E)  be  the  directed  graph  whose  vertex  set  is 
V  -  [1,2,..,N}.  Each  edge  (i,j)  f  E.   is  assigned  a  nonnegative  weight  c    A 
tour  t  f  J"  is  a  cycle  which  passes   through  each  vertex  exactly 

once.  Tour  t  =  (i,  ,i„,...,  1  ,1,)  can  be  given  the  ordered  pair  representation 
12       n   1 

t'  =  [(1^,12).  (12,13)...-,  ^^n-l'^n^'  *^^n'  H"*^  ^  ^'  '^^    traveling- salesman 

problem  is  to  find  a  tour  of  minimum  weight,  i.e.,  find  t  t  J"  so  as  to 


mm 


S     c,  (5) 


t'€j'  (i,j)€t'  ^J 

If  we  think  of  the  vertices  as  cities  and  the  edge  weights  as  the  distances  between 
cities,  the  problem  (5)  is  to  start  at  some  home  city,  visit  all  the  N-1  other 
cities  and  return  home  in  the  minimum  total  distance. 

Consider  the  following  dynamic  programming  algorithm  [14]  for  (5). 
Let  f(S,j)  denote  the  weight  of  a  minimum  weight  path  (subgraph)  which  starts 


at  (contains)  vertex  1,  visits  (contains)  allvertices  in  S  E  [1,2,3,...,  N}  , 

and  terminates  at  vertex  j  €  S.   Then,  for  all  S  ^   0,we  have  the  following 

functional  equations  of  a  dynamic  programming  algorithm  for  the  traveling-salesman 

problem 

f(S,j)  =  min  [c.  +  f(S-j,i)l  i  ^  S-j]  (6) 

with  the  boundary  condition 

f(0,-)  =  0  (7) 

Notice  that  (6)  and  (7)  are  equivalent  to  (1)  and  (2)  with 

Yq  =  (0,-),  y  =  (s,j)  €  n  =  {[(s,j)|j€  s  c  [2,3,...,  n]]  u  ({1,2,...,  n1,i)}, 

y'  =  (S'-j,  i)  ,  d  =  j,  T(y',d)  =  T((S-j,i),j)  =  (S,j),  [(y '  ,  d)  |T(y' ,d)=  y] 

=  {((S-j,l),J)l  i  ^  S-j],  h(f(y'),  y',d)  =  f(S-j,i)  +  c . . ,  and  %^   =  0. 

The  major  roadblock  to  solving  even  moderate  size  (N  s  20)  traveling 

salesman  problems  by  the  dynamic  programming  algorithm, (6)  and  (7),  is  the 

excessive  high-speed  computer  storage  requirement.   The  storage 

bottleneck  occurs  approximately  halfway  through  the  recursion  when  n+1  =  |s|  = 

[n/21, since  the  evaluation  of  f(S,j)  by  (6)  requires  high  speed  access  to 

n(    )   values  of  f (S-j, i).  For  N  =  20,  this  amounts  to  a  high-speed  storage 

requirement  of  92,378  locations.  However,  we  can  reduce  these  requirements  by  fathoming 
certain  states,  thereby  expanding  the  range  of  traveling-salesman  problems 

which  can  be  solved  by  dynamic  programming. 

An  upper  bound  on  (5)  is  easily  obtained  as  the  weight  of  any  tour 

t  €  J"'.   For  example,  we  could  take  the  minimum  of  the  weights  of  i)  the  tour 

(1,2,...,  N-1,  N,l)  and  ii)  the  (nearest-neighbor)     tour  constructed  by 

connecting  vertex  1  to  vertex  k  where  c,   ^  c   Vj  ^  1,  then  connecting  vertex 

Ik    IJ 

k  to  vertex  m  where  c   ^  c,  .  V  j?ti,  j  ^  k,  and  so  on  until  all  vertices  in  [1,2,..,N} 
are  connected,  and  finally  connecting  the  last  vertex  to  vertex  1.   Alternatively,  we 
could  take  the  weight  of  a  tour  generated  by  some  efficient  heuristic  as  our 
upper  bound  1(. 
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The  lower  bound   j2,(S,j)  on  the  weight  of  a  path  j  ~*  1  which  includes 
all  vertices k  ^  S  is  also  easily  obtained.   Since  the  classical  assignment 
problem  is  a  relaxation  of  (5)  we  can  set  i.(S,i)  equal  to  the  value  of  the 
minimum  weight  assignment  to  an  assignment  problem  whose  weight  (cost)  matrix 
C  is  obtained  from  C  by  deleting  all  rows  it  {l  U  (S  -  j)}  and  columns  j  €  S. 
A  relaxation  of  the  traveling-salesman  problem  which  can  be  solved  by  a  "greedy" 
algorithm  is  the  minimum  weight  1  -  tree  problem  which  is  a  minimum  (weight) 
spanning  tree  on  the  vertex  set  [2,...,  Nj  with  the  two  edges  of  lowest  weight 
adjoined  at  vertex  1  [  15,16],   Other  relaxations  have  been  discussed  in  [  3,5,34], 

Suppose  that  we  have  calculated  the  bounds  ^  and  i(S,^).      Then 
Theorem  1  can  be  stated  for  the  traveling-salesman  problem  as  follows: 
Proposition  2.      T-S  Fathoming  Criterion:   If 

f(S,j)  +  £(S,j)  >  U  (8) 

then  any  tour  t  t  ^  which  contains  a  path  between  vertex  I   and  vertex  j_   that 
connects  all  vertices  in  S  -  j  cannot   be  a  minimum  weight  tour,  i.e.  ,  Jt  ^  J^*. 

The  use  of  Proposition  2  is  demonstrated  on  the  following  simple 
numerical  example. 
4.2.1  Example:   Wagner  [34,  p. 472] 

Consider  the  directed  graph  .J'  on  the  vertex  set  {l,2,...,  5} 
whose  weight  matrix  C  is 


C  = 


L 

10 

25 

25 

10 

1 

L 

10 

15 

2 

8 

9 

L 

20 

10 

14 

10 

24 

L 

15 

10 

8 

25 

27 

L 

where  L  €]R   is   sufficiently  large. 


The  calculations  involved  in  the  solution  of  the  functional  equation 
(6)  and  (7),  are  summarized  in  Table  I.   The  weight  of  the  optimal  tour 
{1,5,2,3,4,1}  is  62. 
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TABLE  I,   Summary  of  Calculations  for 
the  Conventional  DP  Solution 
of  the  Traveling-Salesman  Example  Problem 


S  =  0 


IS   =  1 


S   =  2 


(s.i)  f(s,.i)  i± 

(0  ,  -  )     0 

Isl  =  3 


( 
( 
( 
( 


f(S,i)  V 


[2 

3 

4} 

[2 

3 

4} 

[2 

3 

4} 

{2 

3 

5} 

[2 

3 

5} 

[2 

3 

5} 

[2 

4 

5} 

[2 

4 

5} 

[2 

4 

5} 

[3 

4 

5} 

[3 

4 

5} 

[3 

4 

5} 

,2) 

55 

4 

,3) 

45 

2 

,4) 

40 

3 

,2) 

43 

5 

,3) 

28 

2 

,5) 

30 

3 

,2) 

47 

4 

,4) 

33 

2 

.5) 

37 

2 

,3) 

61 

4 

,4) 

55 

3 

,5) 

59 

3 

11^ 

([2], 2) 
([3], 3) 
([4],  4) 
({5}, 5) 


f(S,1)  jj^ 

10  1 

25  1 

25  1 

10  1 


1S|=4 


([2, 3, 4, 5}, 2)  65  4 

({2, 3, 4, 5}, 3)  57  2  or  4 

([2, 3, 4, 5}, 4)  48  3 

({2, 3, 4, 5], 5)  55  3  or  4 


(s,j 

2 

f(s,j) 

i* 

2,3} 

,2) 

34 

3 

2,3} 

,3) 

20 

2 

2,4} 

,2) 

35 

4 

2,4} 

,4) 

25 

2 

2,5} 

,2) 

18 

5 

2,5} 

,5) 

12 

2 

3,4} 

,3) 

49 

4 

3,4} 

,4) 

45 

3 

3,5} 

,3) 

35 

5 

3,5} 

,5) 

35 

3 

4,5} 

,4) 

37 

5 

4,5} 

,5) 

40 

4 

iS   =  5 


({1,2,3,4,5],!) 


f(S,.1)  J^ 
62    4 
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We  next  solve  this  example  problem  with  (6)  and  (7),  incorporating 
the  fathoming  criterion  (8).   The  upper  bound  V.    is  62:   the  minimum  of  the 
weight  (65)  of  tlie  tour  (1,2,3,4,5,1)  and  the  weight  (62)  of  a  nearest- 
neighbor  tour  (1,5,2,3,4,1).   The  lower  bounds  £(S,j)  will  be  calculated  by 
solving  the  assignment  problem  relaxation. 

For  |s|  =  1,  we  have  the  following  calculations: 

(s,.i)        f(s,.i)       i*       ^(s,i) 

([2], 2)  10  1  55 

({3},  3)  25  1  42 

({4}, 4)  25  1  40 

({5}, 5)  10  1  50 

Notice  that  we  can  fathom  states  {  ({  2}  ,  2)  ,  ({  3}  ,3)  ,  ([  4}  ,4)} 
immediately  by  (8),  since  we  have 

f(s,j)  +  ji(s,j)  >  U 

Therefore,  the  maximum  high  speed  storage  for  | S |  =  2  is  1 
location  as  opposed  to  4  locations  in  the  conventional  DP  approach  --  only  in- 
formation on  state  ({5j,5)  is  relevant  to  the  calculation  of  f(S,j)  for  |s|  =  2. 

Furthermore,  by  Corollary  1  we  can  fathom  states  {({2, 3}, 2),  ([2, 3}, 3), 
([2, 4], 2),  ([2, 4], 4),  ({2, 5}, 5),  ({3, 4}, 3),  ({3, 4], 4),  ({3, 5}, 5),  ({4, 5}, 5)} 
before  evaluating  either  f(S,j)  or  i(S,j)  since  they  clearly  could  not  lead 
to  optimal  tours.   Therefore,  for  |S|  =  2  only  3  of  the  12  possible  states 
remain.   The  calculations  for  these  states  are  presented  below: 

(s,i)  f(s,i)  i*  -e(s,i) 

({2, 5},  2)        18  5  44 

([3, 5}, 3)        35  5  31 

({4, 5}, 4)        37  5  28 
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Therefore,  we  can  fathom  states  {([3, 5}, 3),  ([4, 5], 4)}  immediately 
by  (8),  so  that  the  maximum  high-speed  storage  for  \s\    =  3  is  only  1  location 
as  opposed  to  the  12  locations  required  in  the  conventional  DP  approach. 

Again,  by  Corollary  1  we  can  also  fathom  all  states  except  for 
states  i  ([2,3,5}  ,3)  ,  ([2, 4, 5}, 4)}  prior  to  the  evaluation  of  f(S,j)  for 
|sl  =  3,  i.e.,  we  only  have  to  evaluate  f(S,j)  for  2  of  the  12  possible  states 
for  |s|  =  3.  The   calculations  for  the  2  remaining  states  are  presented  below: 

(s.i)       f(s.1)       ii       ^(s,i) 

([2, 3, 5}, 3)     28  2  34 

■   ([2, 4, 5}, 4)     33  2  32 

Therefore,  we  can  fathom  state  ({2, 4, 5}, 4)  by  (8),  so  that  the 
maximum  high-speed  storage  requirement  for  1S|  =  4  is  1  location  as  opposed 
to  the  12  locations  required  in  the  conventional  DP  approach.   However, 
we  don't  even  have  to  store  f([2,3,5},3)  because  by  Corollary  1  the  optimal 
solution  must  be  (1,5,2,3,4,1)1 

Notice  that^  although  the  computation  of  the  lower  bound  j^(S,j) 
required  additional  calculations  in  excess  of  those  required  in  the  convential 
DP  solution,      this  was  offset  by  the  reduction  in  the  number  of  calculations 
as  a  result  of  the  states  (S,j)  which  were  fathomed  by  the  Corollary  1  prior  to 
the  evaluation  of  f(S,j). 

The  drastic  reductions  in  both  storage  and  computational  requirements 
which  were  effected  by  employing  the  fathoming  criteria  with  the  DP  algorithm 
are  summarized  in  Table  II. 

Reference  to  Table  II  reveals  that  the  high-speed  storage  requirement 
was  reduced  by  11/12  and  the  low-speed  storage  requirement  was  reduced  by  26/33 
when  the  fathoming  criterion  were  used  with  DP  in  the  solution  of  this  example 
problem!   However,  this  was  partly  attributable  to  the  very  good  (in  fact,  optimal) 
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TABLI';    Tl.       Summary   of    Reduction    In    IJoth    Storage   and 

Computational  Requirements  Effected  by  Employing 
Fathoming  Criteria  in  the  DP  Solution  of  the 
Traveling-Salesman  Example  Problem. 

Conventional  DP  Solution 


kJ 


1 

2 
3 
4 
5 


No. 

of 

States 

for 

whi 

ch  f(S,j) 

is 

evaluated 

4 

12 

12 

4 

Evaluation  of  f(S,i) 
No.  of     No.  of 


Additions 

0 

12 

24 

12 

4 


Comparisons 
0 
0 
12 
4 
1 


E  = 

max  high-speed 
storage  requirement 

total  low-speed 
storage  requirement 


33 


12 


33 


48 


17 


P.P.  with  Fathoming  Criterion 


M 

1 
2 
3 
4 
5 

L  = 


No.  of  State's 
for  which  f(S,j) 
is  evaluated 

4 

3 

2 


Max  no.  of 
unfathomed 
states 

1 

1 

1 


Evaluation  of  f(S,1) 
No.  of     No.  of 
Addi  tions   Comparisons 


0 
3 
2 


0 
0 
0 


No.  of 
bounds  £(S, 
calculated 

4 

3 

2 


max  high-speed  storage  requirement  = 
total  low-speed  storage  requirement 


1 
7 
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initial  upper  bound.   Only  further  computational  experience  with  the  approach 
can  indicate  if  it  is  competitive  with  other  relatively  efficient  algorithms, 
such  as  [ 16,171  for  the  traveling-salesman  problem.   In  anv  case,  the 
approach  will  significantly  enlarge  the  range  of  traveling-salesman  problems 
which  can  bo  solved  with  dynamic  programming. 

4 ■ 3  Examp 1 e  3 :   The  Nonlinear  Knapsack  Problem  [26] 

The  nonlinear  k napsack  problem  (NKP)  can  be  stated  as  follows: 

N 
find  X  €  IR  so  as  to 

"^  N 


maximize  Z-i     r.(x.) 

(9) 


j=l  J  J' 


subject  to    N 

E  g..(x.)^b,         1=1, 2,... ,M 
j=l   ^J   J     ^ 

X.  fc  S.  i  =  1  ,2 N 

whare  (Vj)  S  =  {0,1,,..,  K  }  and  r  :  S  -  K  is  nondecreasing  with 
r(0)  =  0.  (Vlj)  g^.:  Sj  -m^  with  g^j(O)  =  0,  and  b  =  (b^.b^,...,  b^^)  >  0. 
The  (NKP)  includes  both  the  classical  (one-dimensional)  knapsack  problem  f3l] 
and  the  multidimensional  0/1  knapsack  problem  [35]  as  special  cases  for  which 
(Vj)  r  (x  )  =  ex    (Vij)  g^.(x.)  =  a. .X.  and  K.  is  taken  as  the  smallest 
integer  such  that  g^ .  (K .  +  1)>  b^  or  (Vj )  K.  =  1,  respectively.   The 
(NKP)  also  includes  a  number  of  other  variants  [8]  of  the  classical  knapsack 
problem  and  the  "optimal  distribution  of  effort  problem"  [20,34],  as  well 
as  various  discrete  nonlinear  resource  allocation  problems  [7,9]  graph- theoretic 
problems  [lO,33]  and  nonlinear  integer  programming  problems  [11,13,32],  as 
special  cases. 

Consider  the  following  (imbedded  state  space)  dynamic  programming 
algorithm,  M6MDP  [26],  for  the  solution  of  (9).   Let  f(n,P)  denote  the  maximum 
objective  function  value  of  an  undominated  feasible  solution  to  (9)  in  which  at  most 
all  of  the  first  n  variables  (x  ,x  ,  ,..,  x  )  are  positive  and  whose  resource 
consumption  vector  does  not  exceed  p  =  O-  p   .  ..,p  ),  i.e.,  (Vi)  P.  > 


1  =  1  S£j<.x.;.  For  n  =  1,2,...,  N,  the  feasible  solution  x  =  (x..  ,  x„ ,  .  .  .  ,  x  )  i< 
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said  to  be  dogilnatgd  by  the  feasible  solution  x  =  (x  ,x  , .  .  .  ,  x  )  if  we  have  both 

1^  r.(x.)  s;  2]  .  ,  r.(x.)  and  2J  i  g   (x.)^S._^  g..(x.)with  strict  inequality 

1  =  1   J   .1      j  =  l   J   J        J-J-   iJ   J     J-J-   ij   J 

holding  in  at  least  one  of  the  (M+1)  inequalities.   For  1  5  n  ^e  N ,  let  F  be  the 

n 

(domain)  set  of  resource  consumption  vectors,  P,  of  all  undominated  feasible 

solutions  (x,  ,x„,...,  x  )  to  the  following  subproblem 
1   z       n 


n 
max  Z-j     r  .  ( X  . ) 


j  =  l  ' 


(10) 


subiect  to 

-'  n 


E  g   (x  )  s  b         i  =  1,2,...,  M 
j  =  l    -^   -' 

x^  €  Sj         .1  =  1,2,...,  n 

Also,  for  0  ^  n  ^  N,  let  V  be  the  (domain)  set  of  resource  consumption 

vectors  g  (k)  =  (g,  (k) ,  g„  (k) , . . . ,  g.,  (k)),  of  all  undominated  feasible  values  of 
n         In       zn  Nn 

X  =  k  fc  S  .   We  can  demonstrate  [26]  that  for  1  ^  n  S  N,  F  C  (V  ©  F   ,)  with 
n        n  n     n   n-1 

F-  =  0 ,  where  (V  ®  F   ,)  denotes  the  set  obtained  by  forming  all  sums  of 

0  n   n-1  ■'  ° 

one  element  of  V  with  one  element  of  F 

n  n-1 

Therefore,  for  1  ^  n  ^  N  we  can  recursively  generate  all  feasible 

candidates  for  F   from  F   ,  and  V  with  the  following  functional  equation  of 
n       n-1      n 

M&MDP : 

f(n,p)  =  [r  (k)  +  f(n-l,  p  -  g  (k))|g  (k)  €  V  A  (P  -g  (k))  €  F   .  A  p  S  b]   (11) 
n  n      n       n        n        n- 1 

with  the  boundary  condition 

f(0,0)  =  0  (12) 

If  the  f(n,P)  corresponding  to  dominated  solutions  are  eliminated  at 
the  end  of  (or  during)  each  stage  n,  then  (11)  and  (12)  will  yield  the  set  of 
objective  function  values  {f(N,P)l  P  €  F  }  of  the  complete  family  of  undominated 
feasible    solutions  to  the  (NKP) .   Notice  that  the  functional  equations,  (11)  and  (12) 
together  with  the  dominance  elimination,  are  equivalent  to  (1)  and  (2)  with  y_  « 
(0,0),  y  =  (n,P)  (z  Q   =   {(n,P)|  P  €  F^,  n  =  0,1,...,  n}  ,  y'  =  (n-1,  P-g^(k)), 
d  =  k,  T(y',d)  =  T((n-1,  p-g^(k)),  k)  =  (n,P)  6  Q,    {(y',d)l  T(y',d)  =  y}  = 
{(n-l,P-g^(k))|g^(k)€  V^  A  (P-g^(k))6  F^_^  A  p  S  b},  h(f(y'),  y',d)  - 
f(n-l,  p-g^(k))+  r^(k),  and  5^  =  0. 
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M6J^DP  is  basically  a  one -dimensional  recursive  search  (with  dominance 

elimination)  over  the  sequence  of  sets  F  F    ..,  F  which  are  all  imbedded  in 

the  M-dimensional    space  B   =  [p]     (Vi)   p.    fc    [0,b   1}.      As   a   result   of   the  "imbedded 

state  space"  approach  [25] ,  M&MDP  is  not  overcome  by  the  "curse  of  dimensionality" 

which  inhibits  the  utility  of  conventional  dynamic  programming  algorithms  on 

higher  dimensional   problems.      However,    both    the  high-speed   storage   requirements 

and  the  computational  requirements  are  directly  related  to  the  length  F  I  of 

n 

the  list  F   of  undominated  feasible  solutions  at  stage  n.   Specifically, 

the  high-speed  storage  requirement  is  on  the  order  of  (2M  +  6)n  where  n  = 

max  ilF   }=  maximum  list  length  and,  although  a  very  efficient  dominance 
n   n 

elimination  scheme  (which  eliminates  sorting)  is  employed  in  M6i4DP,  the  com- 
putational time  tends  to  increase  exponentially  with  "n  --  in  fact,  problems 
in  which  n  s  10,000  might  consume  hours  of  computer  time.   Fortunately,  we 
can  employ  the  fathoming  and  relaxation  results  of  S  3  (with  minima  replaced 
by  maxima,  upper  bounds  replaced  by  lower  bounds  ,and  vice  versa)  to  great 
advantage  in  reducing  the  list  lengths. 

Let  =^  be  a  lower  bound  on  the  value  of  an  optimal  solution  x^'<  €  S* 
to  the  (NKP).   For  any  (n,P)  6  Q,  f(n,p)  is  the  value  of  an  optimal  solution 
6'  =  x'  =  (x^*^  x'^*,    ...,  x'*)  €  A*  (n,P)  to  subproblem  (10)  f  or  b  =  p .   C 
sider  the  following  subproblem  for  any  (n,P)  €  Q; 

Find  (x    ,  X     ...,  x.,)  €  IR  ""  so  as  to 
n+i   n+z        N     + 

N 

max   Zj   r . (x  .  ) 


on- 


(13) 


j=n+l   J   J 

subject  to 

N 

S   g. .(x.)  S  b.-p.       i  =  1,2,..  .  ,  M 
j=iH-l   ^J   J      ^   ^ 

X.  €  S.  j  =  n+1,  n+2,  .  .  .  ,  N 

J     J 


•  18- 


Let  X(n,P)  denote  the  set  of  all  feasible  solutions  to  subproblem 

(13).   Clearly,  for  6  =  x  =  (x_^^  ,  x  _^„ ,  ...  ,  x.,)  €  X(n,P),  6 '6  =  x'x  = 

n+  i   n+z        IN 

(x'*,  x'*,...,  x'*,  X  ,,,  X  ,„,...,  x„)  is  a  feasible  solution  to  the  (NKP)  and  we 
1     z        n    n+1    n+z       N 

have 

n  N  N  N 

E  r.(x'*)  +   S  r.(x.)  =  f(n,P)  +   S  r .  (x . )  ^   S  r.(x*)  for  any  x*  t  S*   (14) 

j=l  ^      ^  j=n+l  ^   ^  j=n+l^   J     j=l  J   J 

where  S  *  is  the  set  of  optimal  solutions  to  the  (NKP). 

Let  u(n,P)  be  an  upper  bound  on  the  value  of  an  optimal  solution 

to  subproblem  (13).   Then, Theorem  1  can  be  stated  for  the  (NKP)  as  follows: 

Proposition  2.       (NKP)  Fathoming  Criterion;   If  for  any  (n,P)  €  Q ,  we  have 

f(n,p)  +  U(n,p)  <^  (15) 

then  3  X  €  X(n,P)  such  that  x'x  €  S*. 

Any  P  t  F   for  which  (15)  holds (for  the  corresponding  (n,P)  €  Q) 

can  be  eliminated  from  F  ,  thereby  reducing  the  lengths  of  the  lists 

k 

F    ,    F  ..,    F  Since  p  €    F     may  give  rise  to      11         (K.   +   1)    members    in   list 

n        n+1  N  n       -^  '^  ,         i 

F    (n+l^k^N)  the    effect   of   fathoming   can  be   significant. 

K 

The  choice  of  bounds  and  relaxations  is  dependent  on  the  form  of 

both  the  return  functions  and  the  constraints  and  is  discussed  in  detail  in 

[22] .   The  objective  function  value  of  any  feasible  solution  to  the  (NKP) 

provides  a  lower  bounds.   One  easily  computed  upper  bound  U(n,P)  for  any 

N 
state    (n,P)   €  fi    is       S     r . (K . ) ;    another   is       min   fp      (b_-g_)]   where 

j=n+l   J      ^  ls:iSM    ■    in     i     i 

p        =  max         (max      {r . (k) /g .  .  (k)}) 

n+lSjSN    ISikSK.    -^  ^J 

J 
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We  note  that  for  linear  problems,  as  a  result  of  our  solution  tree  structure, 
linear  programming  relaxations  can  be  used  to  particular  advantage.   That  is, 
in  our  procedure  there  is  no  backtracking  and  the  same  LP  relaxation  of  (13) 
applies  to  all  (n,3)  €  Q;  with  only  the  (b-p)  vector  changing  value.   Notice 
that  for  fixed  n  the  dual  objective  function  value  of  any  dual  feasible  solution 
of  this  relaxation  is  an  upper  bound  for  all  (n,3)  ^  F  .   Also,  the  primal  LP 
relaxation  for  n+1  is  equivalent  to  that  for  n  with  the  column  corresponding 
to  n+1  deleted,  i.e.,  the  LP  relaxations  are  "nested". 

The  "hybrid"  (M&MDP  with  fathoming)  algorithm  was  coded  in  FORTRAN  IV 
and  applied  to  several  of  the  "hard"  linear  problems  which  were  solved  with 
M6J4DP  [26].   The  computational  experience  with  both  the  hybrid  algorithm  and 
the  straight  M&MDP  are  presented  in  Table  III.   The  times  presented  are  the 
execution  (CPU)  times  in  seconds  of  the  codes  on  M.I.T.'s  IBM  370-165. 

In  all  problems  of  Table  III,  the  decision  variables  were  sequeneed 

in  nonlncreaslng  order  of  r.(l)  for  input  to  both  algorithms.   The  simple 

min  {p   (b.-p  )}upper  bounds  were  used  in  the  hybrid  algorithm.   When  Problem  8 
l^iSM   ^"^11 

was  solved  with  the  LP  relaxations  employed  at  every  state  in  the  hybrid  algorithm 

the  max   F  I  =  3  and  the  execution  time  was  4.99  seconds  --  Since   0 

n'  n' 

3 
is  a  member  of  the  F   list  the  savings  were  on  the  order  of  10   (2052  vs  2)  and 

the  computational  savings  were  on  the  order  of  10   (150.00  seconds  vs.  4.99 
seconds)  with  the  hybrid  algorithm. 

Reference  to  Table  III  reveals  the  dramatic  reduction  in  both  com- 
puter storage  requirements  (which  are  a  function  of  max  If  I)  and  computation 

n'  n' 

time  which  were  effected  by  incorporating  the  fathoming  criteria  and  relaxa- 
tions within  M&MDP.   In  fact,  on  the  basis  of  the  additional  computational 
experience  reported  in  [22],  the  hybrid  algorithm  appears  to  be  competitive  with 
even  the  best  available  linear  0/1  integer  programming  algorithm  for  this  special 
case  of  the  (NKP)  and  the  hybrid  algorithm  has  the  ability  to  solve  a  much  broader 
class  of  problems. 
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TABLE  III.   Computational  Comparison  of  the  Hybrid 
Algorithm  vs.  M&MDP 


Problem  Problem  Size 

No.  N    M     K 

4 

5 

6 

7 

8 
12 
13 


15 

10 

20 

10 

28 

10 

39 

5 

50 

5 

34 

5 

50 

5 

M&MDP 

Hybrid 

Algorithm 

execution 

execution 

max  F 

n  n 

time 
(sec.  ) 

max  F 
n  n 

time 

time  (sec. ) 

391 

6.36 

16 

0.29 

526 

6.26 

22 

0.31 

814 

24.18 

62 

0.81 

1043 

11.27 

51 

0.92 

2053 

150.00 

49 

1.33 

540 

8.28 

29 

0.38 

571 

12.48 

12 

0.42 
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5.   Discussion 


It  has  been  demonstrated  that  both  the  computer  storage  require- 
ments and  also  the  computational  requirements  of  dynamic  programming  algorithms 
can  be  significantly  reduced  through  the  use  of  branch-and-bound  methods. 
We  envision  that  this  hybrid  approach  may  open  the  doorways  to  the  solutions 
of  many  sizes  and  classes  of  problems  wViich  are  currently  "unsolvable"  by 
conventional  dynamic  programming  algorithms. 
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